W6-W8. Multivariable Differentiation
1. Theory
1.1 Functions of Several Variables
1.1.1 Domains, Ranges, and Graphs
Most functions you encounter in calculus depend on a single variable. But in science and engineering, quantities almost always depend on several things at once. The temperature inside a room depends on both the coordinates
A function of several variables is a rule that assigns a unique real number to each point in some subset of
The variables
By convention, if no domain is specified, the natural domain is the largest subset of
Finding domains: To find the domain, identify all restrictions:
- Square roots require the argument to be
- Logarithms require a strictly positive argument
- Denominators must be nonzero
Example: For
The graph of
1.1.2 Interior Points, Boundary Points, Open and Closed Regions
When working with multivariable functions, we need to be precise about the structure of their domains.
Let
is an interior point of if there exists a ball of positive radius centered at that lies entirely within . The collection of all interior points is the interior of . A region is open if it consists entirely of interior points. is a boundary point of if every ball centered at contains both points inside and points outside . A region is closed if it contains all its boundary points.- A region is bounded if it fits inside some ball of finite radius; otherwise it is unbounded.
1.1.3 Level Sets
Visualizing a function of two variables as a 3D surface is often difficult. A more practical tool is the level curve.
The set of all points
- For
: a level set is called a level curve (or contour line). - For
: it is called a level surface. - For
: it is called a level hypersurface.
Think of contour lines on a topographic map: each line connects points of equal altitude. Similarly, level curves of
Example: For
1.2 Limits and Continuity for Functions of Several Variables
1.2.1 The Limit Definition
The concept of limit extends naturally to multiple variables, but with an important twist: in
Definition. Let
Here,
1.2.2 Properties of Limits
These properties mirror those from single-variable calculus. If
for any (provided ) for (for when is even)- Composition: If
is continuous at , then
Example:
1.2.3 Two-Path Test for Non-existence of a Limit
The most powerful tool for showing a limit does not exist is the Two-Path Test:
If
The trick is to substitute specific paths (straight lines
Example: Show
Along the parabola
1.2.4 Continuity
A function
is defined, exists, .
A function is continuous if it is continuous at every point of its domain.
Continuity of compositions: If
1.3 Partial Derivatives
1.3.1 Definition
When a function depends on several variables, we can ask: how does it change if we vary only one variable while holding the others fixed? This is the idea of a partial derivative.
Definition. The partial derivative of
Geometrically,
Similarly, the partial derivative with respect to
How to compute: To find
1.3.2 Second and Higher Order Partial Derivatives
Partial derivatives are themselves functions and can be differentiated again. For
— differentiate with respect to twice — differentiate with respect to twice — first , then — first , then
Clairaut’s (Schwarz’s) Theorem: If
1.3.3 Harmonic Functions and Laplace’s Equation
A function
Harmonic functions appear in electrostatics, heat conduction, fluid flow, and many other physical contexts.
Example:
1.3.4 Differentiability and Linearization
A function
Sufficient condition: If
Important implications:
- Differentiability
Continuity (but not vice versa) - Existence of partial derivatives alone does NOT guarantee differentiability
The linearization of
1.3.5 Chain Rule
The chain rule extends to functions of several variables.
Case 1: If
Case 2: If
General Version: If
1.3.6 Implicit Differentiation
If a relation
More generally, if
1.4 Directional Derivatives and Gradients
1.4.1 The Directional Derivative
Partial derivatives measure the rate of change along coordinate axes. But what if we want to know how fast
Definition. The directional derivative of
This is the rate of change of
Important: The direction vector
1.4.2 The Gradient Vector
Definition. The gradient of
For
Key Theorem: If
The directional derivative equals the dot product of the gradient and the direction vector. This elegant formula replaces the limit definition for computation.
1.4.3 Geometric Interpretation of the Gradient
Since
increases most rapidly in the direction of (when ), with rate . decreases most rapidly in the direction of (when ), with rate . has zero rate of change perpendicular to (when ).
Moreover, the gradient is always normal (perpendicular) to level curves (or level surfaces): at any point
1.5 Extreme Values for Functions of Several Variables
1.5.1 Local and Global Extrema
Definition. Let
is a local maximum if for all in some open ball around . is a global (absolute) maximum if for all .- Similarly for local minimum and global minimum.
1.5.2 Critical Points and the First Derivative Test
Definition. An interior point
First Derivative Test: If
Warning: A critical point need not be a local extremum — it could be a saddle point.
Definition. A critical point
1.5.3 The Second Derivative Test (Hessian)
For
Define the discriminant (Hessian):
Suppose
- Local maximum at
: if and . - Local minimum at
: if and . - Saddle point at
: if . - Test inconclusive: if
.
1.5.4 Finding a Function from Its Partial Derivatives
Sometimes we know
- Integrate
with respect to : , where is an unknown function of . - Differentiate the result with respect to
: compare with the given to find . - Integrate to find
, then write the final answer.
Alternatively, start by integrating
1.6 Gradient Descent Method
One of the most important applications of gradients is the gradient descent method — the backbone of modern machine learning.
1.6.1 The Optimization Problem
We want to solve:
In practice, this problem may be impossible to solve analytically (the function may be high-dimensional, non-convex, or have many local minima). Numerical algorithms are needed.
1.6.2 The Algorithm
The gradient descent (GD) method generates a sequence of points:
The idea is simple: the gradient
Key parameters:
is the learning rate (or step size): too large → divergence; too small → very slow convergence.- Stopping criteria:
(gradient is small), (steps are small), or maximum iterations reached.
Caution: In non-convex problems, gradient descent may converge to a saddle point rather than a local minimum, depending on the starting point
Example (2D quadratic): Let
Example (4D quadratic): Let
Convergence in non-convex cases: For non-convex
1.7 Constrained Optimization and Lagrange Multipliers
1.7.1 The Problem of Constrained Extrema
Often we want to optimize a function not freely, but subject to constraints. For example: maximize area given a fixed perimeter, or find the point on a surface closest to the origin.
Method of substitution: Sometimes we can express one variable in terms of others using the constraint and substitute, reducing the problem to an unconstrained one. This works well for simple constraints.
1.7.2 Lagrange Multipliers (One Constraint)
When substitution is inconvenient, the method of Lagrange multipliers provides a systematic approach.
Theorem: Suppose
Geometric intuition: At a constrained extremum, the level curves of
The Lagrangian function:
Setting
Practical procedure for
1.7.3 Multiple Equality Constraints
For
The system to solve becomes:
1.7.4 Linear Programming Problem
A linear programming (LP) problem is a special case of constrained optimization where both the objective function and the constraints are linear:
Here the constraints are inequalities rather than equalities. The feasible set (the region satisfying all constraints) is a convex polytope — a polygon in 2D, a polyhedron in 3D, etc.
Key facts:
- The optimal value of a linear objective over a convex polytope is always attained at a vertex (corner point) of the feasible region.
- The simplex method systematically moves from vertex to vertex along edges, improving the objective at each step.
- Lagrange multipliers generalize to KKT (Karush–Kuhn–Tucker) conditions for inequality-constrained problems: at an optimal point
, there exist multipliers (one per inequality constraint) such that - For LP, the KKT conditions are both necessary and sufficient for global optimality.
Example: Maximize
1.8 Taylor’s Formula for Functions of Several Variables
1.8.1 Motivation and Derivation
Just as single-variable functions can be approximated by polynomials (Taylor polynomials), multivariable functions can also be approximated by multivariate polynomials. This is useful for error estimation, optimization, and analysis near critical points.
For
In general, the
Applying the single-variable Taylor formula to
1.8.2 Taylor’s Formula
Taylor’s Formula for
The remainder
Special case — quadratic approximation at the origin (when all first derivatives vanish at
2. Definitions
- Function of Several Variables: A rule assigning a unique real number
to each point in a domain . - Domain: The largest set of inputs for which the function produces real values.
- Interior Point: A point
such that some open ball centered at lies entirely within . - Boundary Point: A point
such that every ball around it contains both points in and points outside . - Open Region: A region consisting entirely of interior points.
- Closed Region: A region containing all its boundary points.
- Level Set: The set
for a constant . - Level Curve: A level set for a function of two variables (
). - Level Surface: A level set for a function of three variables (
). - Limit:
if approaches from every direction as . - Two-Path Test: If
has different limits along two paths to the same point, the limit does not exist. - Continuity:
is continuous at if is defined and . - Partial Derivative:
; rate of change of with , holding all other variables fixed. - Clairaut’s Theorem: For smooth functions, mixed partial derivatives are equal:
. - Harmonic Function: A function satisfying the Laplace equation
. - Differentiable Function:
is differentiable at if the change can be well-approximated by its partial derivatives (satisfying the error condition with ). - Linearization: The best linear approximation
. - Gradient: The vector
of all partial derivatives. - Directional Derivative:
; rate of change in direction (unit vector). - Critical Point: An interior point where
or some partial derivative does not exist. - Saddle Point: A critical point that is neither a local max nor a local min.
- Hessian / Discriminant:
; used in the Second Derivative Test. - Lagrange Multiplier: A scalar
such that at a constrained extremum. - Lagrangian Function:
. - Gradient Descent: An iterative algorithm
for minimizing . - Learning Rate: The step size
in gradient descent. - Taylor’s Formula (Multivariable): A polynomial approximation of
near a point using partial derivatives up to order , plus a remainder term .
3. Formulas
- Euclidean Norm:
- Partial Derivative (definition):
- Chain Rule (1 parameter):
- Chain Rule (2 parameters):
, - Implicit Differentiation (
): - Implicit Differentiation (
): , - Linearization:
- Gradient:
- Directional Derivative:
(where ) - Max rate of increase:
in direction - Hessian / Discriminant:
- Second Derivative Test: local max if
; local min if ; saddle if - Lagrange Condition (1 constraint):
and - Lagrangian (1 constraint):
- Lagrangian (
constraints): - Gradient Descent:
- Taylor’s Quadratic (at
): where ,
4. Practice
4.1. Find and Sketch the Domain (Lab 6, Task 1)
Find and sketch the domain for each function:
(a)
(b)
(c)
Click to see the solution
(a) Need
(b) Need
(c) Need
Answer: (a)
4.2. Domain Homework (Lab 6, Homework Problem 1)
Find and sketch the domain for each function:
(a)
(b)
Click to see the solution
(a) Need
(b) Need
Answer: (a)
4.3. Sketch Level Curves (Lab 6, Task 2)
Sketch the level curves
(a)
(b)
(c)
Click to see the solution
(a)
(b)
(c)
Answer: (a) Parallel lines; (b) Lines through origin; (c) Hyperbolas
4.4. Level Curves Homework (Lab 6, Homework Problem 2)
Sketch level curves
(a)
(b)
Click to see the solution
(a)
(b)
Answer: (a) Concentric circles; (b) Concentric ellipses
4.5. Find Limits (Lab 6, Task 3)
(a)
(b)
(c)
Click to see the solution
(a) As
(b) Factor: numerator
(c) Let
4.6. More Limit Problems — Homework (Lab 6, Homework Problem 3)
(a)
(b)
(c)
(d)
Click to see the solution
(a) Factor denominator:
(b) Let
(c) Let
(d) Along
4.7. Show No Limit Exists (Lab 6, Task 4)
Show that the following have no limit as
(a)
(b)
Click to see the solution
(a) Along
(b) Along
Answer: Different paths give different limits in both cases.
4.8. Show No Limit Exists — Homework (Lab 6, Homework Problem 4)
Show that no limit exists as
(a)
(b)
Click to see the solution
(a) Along
(b) Along
Answer: Both functions have different limits along different paths.
4.9. Define for Continuity (Lab 6, Task 5)
Define
Click to see the solution
In polar coordinates:
Answer:
4.10. Test Continuity — Homework (Lab 6, Homework Problem 5)
Test for continuity at
(a)
(b)
(c)
Click to see the solution
(a) Polar:
(b) Polar:
(c) Along
4.11. Evaluate Limits (Lab 7, Task 1.1)
Evaluate
Click to see the solution
In polar coordinates:
Answer:
4.12. Evaluate a Limit via Squeeze (Lab 7, Task 1.2)
Evaluate
Click to see the solution
By AM-GM:
Answer:
4.13. Existence of Limit — Three Variables (Lab 7, Task 1.3)
Study the existence of the limit at
Click to see the solution
The plane
Answer: The limit does not exist (domain excludes the plane
4.14. Limit of Three-Variable Function at a Point (Lab 7, Task 1.4)
Study the existence of the limit at
Click to see the solution
Along
Along
Different limits
Answer: The limit does not exist.
4.15. Compute Partial Derivatives (Lab 7, Task 2.1)
Compute
Click to see the solution
Let
Answer:
4.16. Partial Derivatives with Composite Terms (Lab 7, Task 2.2)
For
Click to see the solution
Answer:
4.17. Partial Derivatives of a Power Function (Lab 7, Task 2.3)
For
Click to see the solution
Let
Answer:
4.18. Check Harmonic Functions (Lab 7, Task 2.4)
Determine whether each function is harmonic:
(a)
Click to see the solution
(a)
(b)
(c)
4.19. Chain Rule: (Lab 7, Task 2.5)
If
Click to see the solution
Answer:
4.20. Chain Rule: Three-Variable Composition (Lab 7, Task 2.6)
If
Click to see the solution
Substituting
Answer:
4.21. Implicit Differentiation: (Lab 7, Task 2.7)
From
Click to see the solution
Answer:
4.22. Implicit Differentiation: , (Lab 7, Task 2.8)
If
Click to see the solution
Answer:
4.23. Directional Derivative (Slope Toward a Point) (Lab 7, Task 2.9)
For
Click to see the solution
Direction:
Answer:
4.24. Directional Derivatives — Lab 8 Exercises (Lab 8, Task 1)
Find the slope at point
Click to see the solution
Answer:
4.25. Directional Derivative — Lab 8 Homework (Lab 8, Homework Problem 1)
Determine
Click to see the solution
At
Answer:
4.26. Directional Derivative of (Lab 8, Task 2)
For
Click to see the solution
Answer:
4.27. Classify Critical Points — Lab 8 Homework (Lab 8, Homework Problem 2)
Find and classify the critical points of:
(a)
(b)
Click to see the solution
(a)
Substitute:
: → saddle. : , → local min.
(b)
Substitute:
: → saddle. : , → local minima.
Answer:
Saddle at
; local min at .Saddle at
; local minima at and .
4.28. Directional Derivative at an Angle (Lab 8, Task 3)
Find the directional derivative of
Click to see the solution
Answer:
4.29. Find Functions from Partial Derivatives (Lab 8, Task 4)
Find
(a)
(b)
(c)
Click to see the solution
(a) Integrate
(b) Integrate
(c) Integrate
Answer: (a)
4.30. Find and Classify Critical Points (Lab 8, Task 5)
Find and classify the critical points of:
(a)
(b)
(c)
Click to see the solution
(a)
: , → local min ( ). : , → local max ( ). and : → saddle points.
(b)
: , → local max. : , → local min. and : → saddle points.
(c)
: saddle (function changes sign nearby). and : local max ( ). and : local min ( ).
4.31. Find and Classify Critical Points (Lab 9, Task 1)
Find and classify the critical points of
Click to see the solution
Key Concept: Use the second derivative test: compute
Compute
:Compute
:Set
and : Since always, we need: : or . : or .
Solving simultaneously gives five critical points:
, , , .
Compute second partial derivatives:
Classify each critical point:
(a) At
: , , . So . Saddle point at .(b) At
: Let . . By symmetry, . . , and . Local maximum at . The value is .(c) At
: By symmetry ( ), also a local maximum with value .(d) At
: . , . , . Local minimum at . The value is .(e) At
: By symmetry, also a local minimum with value .
Answer: Five critical points:
: saddle point. (same sign): local maxima with value . (opposite sign): local minima with value .
4.32. Maximum Volume of a Lidless Box (Lab 9, Task 2)
A rectangular box without a lid is to be made from
Click to see the solution
Key Concept: Use Lagrange multipliers. The objective function is the volume
Set up:
- Objective:
. - Constraint:
.
- Objective:
Lagrange equations (
, with ):Solve by symmetry: Multiply first equation by
, second by , third by : So (assuming ).From
: .From
and : .So
.Apply constraint:
, : Thus , .Maximum volume:
.
Answer: The maximum volume is
4.33. Extreme Values on a Circle (Lab 9, Task 3)
Find the extreme values of
Click to see the solution
Set up:
- Objective:
. - Constraint:
. - Lagrange function:
.
- Objective:
Lagrange equations:
Solve:
- From
: either or . - From
: either or .
Case 1:
. Then . Constraint: , so . Points: , .Case 2:
. Then . Constraint: , so . Points: , .Case 3:
and — impossible on the unit circle.- From
Classify: Minimum value
at ; maximum value at .
Answer: Minimum is
4.34. Points on a Sphere Closest/Farthest from a Point (Lab 9, Task 4)
Find the points on the sphere
Click to see the solution
Key Concept: Minimizing/maximizing the distance is equivalent to minimizing/maximizing the squared distance
Set up:
- Objective:
. - Constraint:
.
- Objective:
Lagrange equations:
Simplify: , , .Solve:
, , (assuming ). So , , .Apply constraint:
.Two solutions:
: . Squared distance . Since , this point lies in the same direction as from the origin — closest point. : — farthest point.
Distance from origin to
is . The sphere has radius .- Closest distance:
. - Farthest distance:
.
Answer:
- Closest point:
, distance . - Farthest point:
, distance .
4.35. Maximum and Minimum of on the Unit Circle (Lab 9, Task 5)
Find the maximum and minimum of
Click to see the solution
- Lagrange equations:
, , . - Solve: From the first two equations:
and . If : . : . Constraint: . Points: and , . : . Points: and , .
Answer: Maximum is
4.36. Second Degree Taylor Polynomial at (Lab 9, Task 6)
Find the second degree Taylor polynomial for
Click to see the solution
Key Concept: Use the Maclaurin formula
- Compute values at
: . . . . . .
- Assemble:
Answer:
4.37. Second Degree Taylor Polynomial at (Lab 9, Task 7)
Find the second degree Taylor polynomial for
Click to see the solution
- Compute values at
: . . . . (since , so ). .
- Assemble with
, :
Answer:
4.38. Estimate Using Second Degree Taylor Polynomial (Lab 9, Task 8)
Use a second degree Taylor polynomial to estimate
Click to see the solution
Key Concept: We expand
Compute values at
: . . . . . .
Second degree Taylor polynomial:
with , : , . , , . .
Estimate:
(For comparison, the exact value is
.)
Answer:
4.39. Evaluate a Limit by Direct Substitution (Chapter 2, Example 1)
Evaluate
Click to see the solution
Key Concept: When the denominator is nonzero at the limit point and the outer function is continuous, use the Composition Rule.
- Evaluate the inner fraction at
: - Apply continuity of cosine:
.
Answer:
4.40. Evaluate a Limit Using the Product Rule (Chapter 2, Example 2)
Evaluate
Click to see the solution
- Split using the Product Rule for limits:
Answer:
4.41. Evaluate a Limit by Direct Substitution (3 Variables) (Chapter 2, Example 3)
Evaluate
Click to see the solution
- Substitute
directly (the expression is continuous at this point):
Answer:
4.42. Evaluate a Limit Using Factoring (Chapter 2, Example 4)
Evaluate
Click to see the solution
Key Concept: Factor the numerator to cancel the indeterminate factor
- Factor by grouping:
- Cancel and evaluate:
Answer:
4.43. Evaluate a Limit by Rationalizing (Chapter 2, Example 5)
Evaluate
Click to see the solution
Key Concept: Multiply by the conjugate to eliminate the indeterminate form.
- Multiply numerator and denominator by
: - Take the limit:
Answer:
4.44. Show a Limit Does Not Exist (Two-Path Test) (Chapter 2, Example 6)
Does the limit
Click to see the solution
Key Concept: If two different paths to
- Along the
-axis ( , ): . - Along
: .
Two different limit values (
Answer: The limit does not exist.
4.45. Show a Limit Does Not Exist (Parabola Paths) (Chapter 2, Example 7)
Show that
Click to see the solution
- Along
(family of parabolas): - This limit depends on
: for it equals ; for it equals .
Answer: The limit does not exist.
4.46. Compute Partial Derivatives (Chapter 2, Example 8)
For
Click to see the solution
(treat as constant): (treat as constant):
Answer:
4.47. Partial Derivatives of a Composite Function (Chapter 2, Example 9)
For
Click to see the solution
(product rule + chain rule): (chain rule only, is constant):
Answer:
4.48. Implicit Differentiation: One Equation (Chapter 2, Example 10)
Assuming
Click to see the solution
- Define
. - Compute
and . - Apply:
.
Answer:
4.49. Implicit Differentiation: Three Variables (Chapter 2, Example 11)
Assuming
Click to see the solution
, , .- Apply:
; .
Answer:
4.50. Chain Rule with One Parameter (Chapter 2, Example 12)
For
Click to see the solution
, ; , .- Chain rule:
. - Substitute
, and simplify:
Answer:
4.51. Chain Rule with Two Parameters (Chapter 2, Example 13)
For
Click to see the solution
- Partial derivatives of
: - Partial derivatives of
, : - Apply chain rule:
Answer:
4.52. Directional Derivative via Gradient (Chapter 2, Example 14)
Find the derivative of
Click to see the solution
- Normalize:
. - Gradient at
: , . - Directional derivative:
Answer:
4.53. Classify Critical Points (Chapter 2, Example 15)
Find and classify the critical points of
Click to see the solution
or ; or .- Critical points:
, , , . , : : → saddle point. : , → local maximum. : , → local minimum. : → saddle point.
Answer: Local max at
4.54. Constrained Optimization by Substitution (Chapter 2, Example 16)
Find the point on the plane
Click to see the solution
- Minimize
with : - Set
: , , so . - Verify minimum:
, .
Answer:
4.55. Lagrange Multipliers — One Constraint (Chapter 2, Example 17)
Find the largest and smallest values that
Click to see the solution
- Set up
with : - Solve:
, giving critical points . - Evaluate:
(max); (min).
Answer: Maximum
4.56. Lagrange Multipliers — Two Constraints (Chapter 2, Example 18)
Find the extreme values of
Click to see the solution
- Set up with multipliers
, : , , . - Solve:
, giving and . - Evaluate:
; .
Answer: Maximum
4.57. Taylor Quadratic Approximation (Chapter 2, Example 19)
Find a quadratic approximation to
Click to see the solution
- Compute derivatives at
: ; . - Quadratic Taylor:
. - Error bound: Third-order derivatives
in absolute value, so
Answer:
4.58. Lecture Exercises — Chain Rule (Chapter 2, Task 1)
Find
(1)
(2)
Click to see the solution
(1)
(2)
With
Answer (1):
4.59. Lecture Exercises — Implicit Differentiation (Chapter 2, Task 2)
Assuming
Click to see the solution
. . At : . . At : . .
Answer:
4.60. Lecture Exercises — Gradient (Chapter 2, Task 3)
Find the gradient at the given point:
(1)
(2)
Click to see the solution
(1) Let
At
At
(2) At
Answer (1):
Answer (2):
4.61. Lecture Exercises — Directional Derivative (Chapter 2, Task 4)
Find the derivative at
(1)
(2)
Click to see the solution
(1) Normalize:
At
(2) Normalize:
Answer (1):
Answer (2):
4.62. Lecture Exercises — Max/Min Directions (Chapter 2, Task 5)
Find the directions of fastest increase and decrease at
(1)
(2)
Click to see the solution
(1) At
Max increase direction:
Max decrease direction:
(2)
Max increase direction:
Max decrease direction:
Answer (1): Max increase:
Answer (2): Max increase:
4.63. Lecture Exercises — Lagrange Multipliers (Chapter 2, Task 6)
Find the points on the ellipse
Click to see the solution
; Lagrange condition: , .- Solve:
, giving , . - Evaluate:
.
Answer: Max
4.64. Lecture Exercises — Taylor Approximations (Chapter 2, Task 7)
Use Taylor’s formula to find quadratic and cubic approximations of
Click to see the solution
- Derivatives at
: ; ; . ; ; . ; ; ; .
- Quadratic approximation (
): - Cubic approximation (
):
Answer: Quadratic: